Variational representation , HCR and CR lower bounds

It should be noted that the requirement of f to be convex in the definition of f -divergence is essential. In Euclidean spaces any convex function can be represented as the pointwise supremum of a family of affine functions and vice versa, every supremum of a family of affine functions produces a convex function. Take f(x) = 12 |x− 1| as an example. We see that it can be written as a pointwise supremum of f1(x) = 1 2(x − 1) and f2(x) = 1 2(1 − x). This remark can be used not only as a geometric interpretation of convex functions but as a definition of convexity. For f -divergences which are convex functions of probability measures, its variational representation amounts to writing it as a pointwise supremum of affine functions.